Modular representation theory is a branch of mathematics, and is the part of representation theory that studies linear representations of finite groups over a field K of positive characteristic p, necessarily a prime number. As well as having applications to group theory, modular representations arise naturally in other branches of mathematics, such as algebraic geometry, coding theory, combinatorics and number theory.

Within finite group theory, character-theoretic results proved by Richard Brauer using modular representation theory played an important role in early progress towards the classification of finite simple groups, especially for simple groups whose characterization was not amenable to purely group-theoretic methods because their Sylow 2-subgroups were too small in an appropriate sense. Also, a general result on embedding of elements of order 2 in finite groups called the Z* theorem, proved by George Glauberman using the theory developed by Brauer, was particularly useful in the classification program.

If the characteristic p of K does not divide the order |G|, then modular representations are completely reducible, as with ordinary (characteristic 0) representations, by virtue of Maschke's theorem. In the other case, when |G| ≡ 0 mod p, the process of averaging over the group needed to prove Maschke's theorem breaks down, and representations need not be completely reducible. Much of the discussion below implicitly assumes that the field K is sufficiently large (for example, K algebraically closed suffices), otherwise some statements need refinement.